It took me a while because of travel and other projects, but I finally released a tar ball containing Debian lenny for the SheevaPlug along with some installation instructions. It should be really easy to install Debian this way, but I'll also make sure that SheevaPlug support will be in the Debian installer for the next release of Debian. For now, check out the tar ball I prepared!

What seems like eons ago, I ordered one of those SheevaPlug Development Kits. After about six weeks, they got around to shipping it, and then, yesterday, after I assume a tortoise had piloted it across the country on broken tricycle, it was finally delivered to me. I booted it up, observed that there was some kind of Ubuntu thing on it, and set myself to correct that problem. Within an hour I had managed to lock myself out. Here is something I should have read beforehand. Thanks to Martin Michlmayr, it is now running Debian and allowing me to log in. Here are some steps to follow if you would like to boot Debian off of a USB stick plugged into your Sheevaplug:

• Grab the tarball from http://people.debian.org/~tbm/sheevaplug/lenny/
• Make an ext3 filesystem on the first partition of your USB stick (ext2 will not work with the current image)
• Mount the USB stick's filesystem, unpack the tarball onto it, then unmount
• Plug the USB stick into the SheevaPlug's full-sized USB port
• Plug the SheevaPlug in, and use minicom or your favorite terminal program to console in over the mini-USB port (don't forget that you need to use the ftdi_sio module instead of usbserial; 115200,N,8,1, no flow control)
• Abort the autoboot, and type the following commands:

  setenv mainlineLinux yes
  setenv arcNumber 2097
  saveenv
  reset
  setenv bootargs_root 'root=/dev/sda1 rootdelay=10'
  setenv bootcmd_usb 'usb start; ext2load usb 0:1 0x0800000 /boot/uInitrd; ext2load usb 0:1 0x400000 /boot/uImage'
  setenv bootcmd 'setenv bootargs $(console) $(bootargs_root); run bootcmd_usb; bootm 0x400000 0x0800000'
  saveenv
  run bootcmd
  

That should be all there is to it. Next I'm wondering if SDIO wireless cards work.

I just noticed Matthew Palmer's Insane/Brilliant idea of the day: he proposes to use large sets of USB serial adapters with a sheevaplug as a cheap serial console server. As part of upstream on cereal, i feel obliged to mention that package as a tool for managing serial console farms like this. It's designed to run in a small footprint, stores timestamped logs for the consoles, supports concurrent remote access, and uses standard unix accounts (usually via ssh) to permit read and/or write access to each port. It has saved me exactly the headaches Matt describes many times. However, i've had trouble getting multiple identical USB serial adapters to persist at standard device file locations across reboot. That is, if you have four pl2303 devices from the same manufacturer, it seems to be a crapshoot which one will be /dev/ttyUSB0 after you restart your system. I could find no distinguishing data in the sysfs to get udev to persistently key off of, anyway. if you know a way to do it, i'd be happy to see it! Depending on how many ports you need, another alternative would be to use a sheevaplug with a multiport USB-to-serial adapter. While i haven't tried this specific hardware, it would remove the need for the hub, and potentially would mean you didn't need any extra power. I'm assuming that this device would give you persistent port naming, but i haven't tried it. Pricewise, it seems to be a win, too: $100 for the SheevaPlug and $100 for the 8-port adapter.Tags: cereal, sheevaplug

I've been talking serial consoles with a couple of the other guys at work: how nice they are to have for machines in the datacenter, how annoying it is that vPro serial-over-LAN doesn't seem to be robust (yet?), and how serial access concentrators are lung-and-kidney expensive (especially when you've got 50-some racks to outfit). This discussion, combined with my ongoing embedded-hardware-fascination lust for a SheevaPlug appears to have spurred my brain into coming up with a Brilliant Idea: tie a SheevaPlug to a pile of USB to serial adapters and use that as your per-rack serial concentrator. Imagine: faffenheimer, a dedicated server you manage for a customer, and located in rack 27 of your DC, has just crashed, and you'd like to know WTF has happened rather than just blindly reboot, but you're in the office 15 minutes away from the DC floor, and the customer's going to want that machine back up and running pretty quickly.

workstation:~/porn$ ssh rack27.serial
rack27:~$ sconsole faffenheimer
[screen session attached]
[minicom running, shows the horror of a kernel crash dump]
[oh look at that, kernel bug]
^A ESC
[pgup pgup]
[enter]
[pgdn pgdn]
[enter]
^A >/tmp/faffenheimer-crash-dump
^A d
rack27:~$ exit
workstation:~/porn$ scp rack27.serial:/tmp/faffenheimer-crash-dump ~
workstation:~/porn$ powercycle rack27 faffenheimer

Shiny! We got a crash dump in a minute or so (rather than having to take phonecam photos of KVM screens in the DC), never had to leave our comfy seat, and the machine's on it's way back up. We're now free to pursue diagnostic activities on that crash dump at our leisure. 10 minutes later, the downtime for faffenheimer that was automatically set when we ran powercycle runs out and Nagios sends us threatening messages. Hmm, something's gone wrong here. Back into the console...

workstation:~/porn$ ssh rack27.serial
rack27:~$ sconsole faffenheimer
[screen session attached]
[Boot is hung waiting for root password after initrd has bombed]
[Type root password]
[Oh look, the root MD appears to have come asunder]
[clickety-click... fixee fixee]
[reboot]

The more I think about this, the more I reckon I'm onto a bona fide winner. The sheevaplug is a powerful ARM-based system with USB/ethernet/SD ports that is packaged literally in it's own power supply wall wart -- it's a plastic box with power plug prongs poking out the side. That's all there is to it. The USB to serial adapter things are likely to be a bit more of a pain, but I've played with enough of them by now to not be too scared. So, you plug the Sheeva into a power socket, plug an Ethernet cable and USB hub into the Sheeva, configure things a bit so that the system knows which serial adapter maps to which machine, and you're away. Oh, and the best bit: the Sheeva apparently draws as little as 2W when idle. A whole datacentre's worth of serial goodness for about a server's worth of power. The cost per rack should be somewhere below AU$250, especially in bulk. Let's see if I can convince work to spring for a Sheeva, a USB hub, and a half dozen or so USB to serial adapters to test this whole thing out. Given that the whole thing looks like it'd cost less than AU$250 (plus my R&D time), I can't imagine it'll be too hard a sell to at least give it a go. Watch this space...

My patch for initial kernel support for the QNAP TS-119 and TS-219 Turbo NAS got accepted and will be in 2.6.30. I can run Debian on a USB disk without any problems but there are some issues left (some severe, some cosmetic):

• SATA: there are long timeouts when initializing the disk. In most cases, the disk is eventually recognized but it can take quite some time. I've no idea why those timeouts happen.
• Flash: I can read from flash just fine but writing doesn't work.
• Parsing the MAC address: even though a MAC address is set in u-boot, the QNAP firmware stores the real MAC address in flash (similar to the TS-109 and TS-209). The TS-109/TS-209 code to read the MAC address doesn't work because the flash on the TS-219 is connected in a different way (SPI).
• SATA lights: the activity light doesn't work.

Unfortunately, I've no idea how to resolve these issues. However, the QNAP TS-119 and TS-219 have started shipping so hopefully someone else will investigate these issues.

Various people were wondering how much power the SheevaPlug consumes exactly. Laurent Guerby, who runs the GCC Compile Farm, posted some data today. According to his measurements with an Energy Monitor 3000, the SheevaPlug uses:

3.1W	just on, Linux booted
3.3W	serial
4.3W	serial + Ethernet
5.4W	serial + Ethernet + 100% CPU
6.0W	serial + Ethernet + USB disk
7.0W	serial + Ethernet + USB disk + 100% CPU

However, these figures should be seen as preliminary and relatively high values since power management for Kirkwood has not been implemented in the kernel yet. In particular, the Kirkwood chip has SATA and PCI-E but they could be turned off since they're not used by the SheevaPlug. This is currently not done in the kernel, so you can expect some improvements in this area.

I received a SheevaPlug this week, an intriguing device that packs incredible power and functionality into a tiny package. As many of you know, I've been doing a lot of work on Debian for the Linksys NSLU2 in the last few years. The NSLU2 is a key reason why ARM has become the third most popular architecture in Debian (after 32 and 64 bit x86), and I believe a main reason is that the NSLU2 is so incredibly cheap. At a price under $100, most people don't think too long and simply buy a device and do something cool with it. The SheevaPlug is being offered at the same price range but offers considerably more. Riku Voipio asked the right question: "What would you do with something approximately 10x more powerful with same prize/size range?" I believe the SheevaPlug is a killer replacement for the NSLU2 and here's why:

• Marvell Kirkwood 6281 CPU at 1.2 GHz with 256 KB L2 cache (compared to a 266 MHz CPU)
• 512 MB RAM (compared to lousy 32 MB)
• 512 MB flash, enough for a basic Debian installation (compared to 8 MB)
• Gigabit Ethernet (compared to 100 Mb) and no proprietary microcode required
• 1x USB and 1x SD (compared to 2x USB: the only advantage the NSLU2 has)
• ... and finally: mini-USB with serial console and JTAG: while Debian on the NSLU2 works well and is stable, we regularly had users whose system would no longer boot and there was no good way to figure out why. The serial console on the SheevaPlug actually allows users to figure out what's going on.

I'm incredibly excited about the SheevaPlug and the first thing I did was to take the device apart and look at the inside. The results can be found in the SheevaPlug image gallery. My next project will be slightly more productive: porting Debian. As I see it, we should support the following three installation variants for the SheevaPlug:

• To an external USB drive or stick.
• To an SD card.
• To the 512 MB flash memory on the device.

The first two should be relatively straight forward, but of course installing to the internal flash memory is particularly interesting given that 512 MB (plus compression) is enough for a basic installation of Debian. Unfortunately, installations to MTD flash are currently not supported in the Debian installer but I hope we can find a volunteer who wants to implement this functionality. My next steps are to put a kernel for the SheevaPlug into the archive and to get a basic installation going. From there we can look at more sophisticated installation options and other functionality.

I received a sample of QNAP's new TS-219 Turbo NAS today. The TS-119 and TS-219 devices are an upgrade to QNAP's TS-109 and TS-209 devices and were announced earlier this month at CeBIT. The new TS-219 uses the same robust case as the TS-209 but offers much more performance:

• Marvell Kirkwood 6281 at 1.2 GHz (instead of Orion 5182 at 500 MHz)
• 512 MB RAM (instead of 128 or 256 MB)
• 16 MB flash (instead of 8 MB)

The numbers I've seen about the Kirkwood CPU suggest that the TS-219 will indeed give a very significant performance boost. So far, the TS-219 looks very nice. My only complaint is that QNAP didn't export the second Ethernet port the Kirkwood chip offers, but I suspect this is because they wanted to stay as close to the original design of the TS-209 as possible. In any case, the TS-219 is a nice machine and I look forward to porting Debian to it. As a first step, this means getting the mainline kernel to run on the device and adding a kernel for Kirkwood to the archive (the latter is needed for a number of other devices based on Kirkwood anyway). Hopefully, the remaining porting work and integration into the Debian installer should be fairly straightforward given that the TS-209 is already supported in Debian. I'm sure I'll find out as I dig into the details... I'll give periodic updates of my progress. In the meantime, I've created an image gallery of the TS-219.

I was not in a work mood tonight and then decided to lose my time reading about timezones (thanks to a Chinese user who wants to add Beijing to the list of timezones for China...which has only one, including the colonies of Tibet and Xinjiang). The list given on Wikipedia is a completelt hilarious time waster, indeed. My favourite one is probably the mention then some parts of British Columbia (Canada) are in UTC-7, namely Most of Peace River Regional District except Fort Ware, Pink Mountain as well as Regional District of East Kootenay and finally (take a breath!) Regional District of Central Kootenay east of the Kootenay River and parts east of Kootenay Lake that are south of and including Riondel (Creston doesn't observe DST), and Columbia-Shuswap Regional District east of the Selkirk Mountains. Canada timezones are really a gem, where one learns that Northwest Territories (one of the largest states) follows UTC-7 except a town named Tungsten and.....a tungten mine. Some other gems in that page are the places that leave in the future at UTC+13 and UTC+14, in Tonga and Kiribati. A other one? Arizona is also funny with the entire state being UTF-7, but the Navajo nation not observing DST, but the Hopi reservation (that's an enclave of the Navajo nation) observing it. That gives an interesting nightmare when travelling over there in summer. And finally, living in Florida probably requires a very high intelligence to understand what time it can be when "east of the Apalachicola River, plus the portions of Franklin County and Gulf County south of the Intracoastal Waterway, west of the Apalachicola River" uses UTC-5 while "west of the Apalachicola River, except for the portions of Franklin County and Gulf County south of the Intracoastal Waterway" use UTC-6. There are other such hilarious thingies on that page. I hope you'll enjoy it as much as I did.

popcon lists approximately 1000 arm/armel Debian installations. There is also listed 864 installations of nslu2-utils, letting us estimate that approximately 85% of debian/arm(el) installations are Linksys nslu2's. It is nicked sympathetically as "The Slug", which pretty accurately describes the performance of nslu2. Still, people have found absolutely amazing amount of ways to use their slugs. What would you do with something approximately 10x more powerful with same prize/size range?

Enter the Marvell SheevaPlug

  What   Slug      SheevaPlug  
  CPU    266Mhz     1.2Ghz      
  Cache  32KB       32+256KB    
  Flash  8MB        512MB       
  MEM    32MB       512MB       
  Net    100Mb      1Gb         

And that's not everything - SheevaPlug comes with SDIO slot and miniusb to be used as a serial console (and JTAG). No soldering needed for hacking.

Some more details on the LinuxDevices article.

For those of you who think that has one port too few of something, or don't like the wall-wart design, Other devices based on kirkwood SoC (which SheevaPlug is based on) are on the way from various ODM/OEM houses.

A teacher in Arizona steals Linux CDs from a student and then accuses a Linux distributor of being a criminal [1]. Even though she had used Linux in the past she didn t believe that software was free. Of course that implies that in the past she had performed actions that she believed were criminal. Neat Little Mac Apps interviews Marshall Kirk McKusick - he describes how the BSD Daemon logo was designed and one of his most significant bugs [2]. OurDelta.org offers MySQL builds with some extra features and support [3]. I was recommended to use their builds by Arjen Lentz of Open Query [4], as one of my clients is going to use the services of Open Query it seems best to use the Our Delta builds if only to get better support from Open Query. The extra features in the Our Delta builds seem interesting, but I m not sure that my client needs any of them at this time. The Global Guerilla blog reports on a man who single-handedly invaded the most heavily guarded power station in Britain and shut it down to protest against new coal power stations [5]. The entire blog is worth reading, the author has a lot of interesting ideas. PhpMyVisites is a free web site analytics system that competes with Google Analytics [6]. I haven t implemented it yet, but it looks promising. It seems that PhpMyVisites is not being updated any more (not even security updates) and the replacement is Piwik [11]. Andrew Lahde was a fund manager who made significant amounts of money by betting on the inability of US mortgagees to repay their debts, he wrote an interesting goodbye letter (Telegraph.co.uk) [7]. He now has a Wikipedia page which gives some interesting background to his career [8]. An Employee of the Financial Times is famous for flaming Andrew [9], I have submitted a comment pointing out that being famous for flaming someone who is more successful than yourself is nothing to be proud of and suggesting that he advocate his own political views when criticising those of others - I doubt that it will get through moderation. It s a pity that Andrew doesn t have a blog, I would like to read more from him. At CCC a paper by Alexander Sotirov, Marc Stevens, Jacob Appelbaum, Arjen Lenstra, David Molnar, Dag Arne Osvik, and Benne de Weger on how to crack the PKI infrastructure used for SSL signing was presented [10]. The root cause is some CAs still using MD5 even though it was broken a long time ago. Updated to note that Piwik is the replacement for PhpMyVisites.

• [1] http://linuxlock.blogspot.com/2008/12/linux-stop-holding-our-kids-back.html


• [2] http://www.neatlittlemacapps.com/show/nlma-episode-048-interview-marshall-kirk-mckusick


• [3] http://ourdelta.org/


• [4] http://openquery.com.au/


• [5] http://globalguerrillas.typepad.com/globalguerrillas/2008/12/journal-climate.html


• [6] http://www.phpmyvisites.us/


• [7] http://www.telegraph.co.uk/finance/financetopics/financialcrisis/3219870/The-astonishing-sign-off-of-a-wealthy-fund-manager-in-full.html


• [8] http://en.wikipedia.org/wiki/Andrew_Lahde


• [9] http://blogs.ft.com/gapperblog/2008/10/yours-truly-angry-mob/


• [10] http://www.phreedom.org/research/rogue-ca/


• [11] http://piwik.org/

The last day of each year sees one of the strangest traditions I've ever noticed. A 40-year-old English-language film of an 80-year-old music hall comedy is played repeatedly on various stations in Germany - and in Austria, Switzerland, Denmark and Australia too. There's not only the film, but edited versions, localised versions, spoofs and other shows based around it too. The film still isn't shown on any English stations, as far as I can tell. I'd never heard of it before getting satellite TV. So, to see what the fuss is about, grab your Dinner for One times (may contain annoying ads if you switch Javascript on) and sit down with the satellite TV (best), or enjoy this description and script from analoguesat, a German-language description or this BBC report which includes this snippet:-

"To the end of his life, Freddie Frinton heartily disliked Germany and the Germans, thanks to his own wartime experience. He refused to allow a German-language version to be made. That is what has led to the extraordinary fact that today the Germans as a nation have embraced a trifling one-act play in English as their all-time favourite entertainment."

Anyway, the last joke is on we English. It's much better than the aging boy bands on our TV at New Year, isn't it? (Updated 2008-01-03 with 1 comment)

These questions came in as comments this week, but I think they're worth a whole post. Peter Swan asked:

"This may seem like a dumb question, but I have not seen anywhere the answer to it. Can a satellite dish be mounted in a loft space or must it be mounted outside?"

There are few dumb questions. DIY dish info is still rather hard to find. You can install a dish behind a window and I've read about special tiles which allow satellite frequencies through on analoguesat's pages about hiding dishes but I don't know anyone using them. A more usual solution is either a hidden location such as a flat roof with baffle boards (dishes only need to see the Clarke belt, which is an elevation of about 20 degrees in England) or to use a non-dish antenna like the digiglobe or the LX2000 pipe. I think it's a really good idea to put the dish somewhere that you can fix it easily. Even the best dish seems to age eventually. I suspect some "professional" installers put the dishes at tops of walls to encourage repair call-outs. Marcus asked:

"Is it possible to receive British satellite television in other countries? I am living in Finland and I was considering purchasing a sky system from the UK and installing it over here. Would it work? Also would I be able to connect a sky set-top box to a different dish?"

Yes, you can get British TV in other countries. Getting external TV like BBC World is easy, but BBC UK services are on Astra 2D, which is focused on the UK. Coverage maps are available on many sites and it looks like Finland isn't well-covered. You'd need a colossal dish. You can connect Sky boxes to other dishes. Sky dishes and Sky LNBs need to go together, sometimes called Skyware, but dishes and boxes are interchangeable. A Sky system will work in Finland, but will only receive services on the other beams like 2A and I doubt Sky will register you at a non-UK address (anyone know for sure?). Also, Sky boxes are horribly crippled for anything other than Sky services and hopefully BBC Freesat will make BBC UK services easier-to-use with standard boxes.

My personal email account was spammed by Jennifer Kirkham-Sandy (Phone 0800 6 121 151) of Total Language Solutions Limited, 3 The Courtyard, New North Road, Exeter EX4 4EP, Devon. Registered number (England and Wales) at the above address: 03933805. They seemed to be using a service called qwertywordmarketing. I've reported it to the hosting company, so we'll see what happens.

Successful Mission, Captain Kirk In 1998, as an undergrad, I encountered what I remember to be my first real university challenge: write a MIPS kernel which had to run on the appropriate emulator. An acceptably decent running kernel should have made the emulator output in the end the Star Trek quote in the title. Now, a lot time after (wow, 10 years, has it been that long?), it's over. Yesterday I defended my PhD thesis. I feel a bit emptied now ...

The problem with going away on these lengthy trips, is I never end up blogging about it while I'm there, so I have to write a behemoth post when I get back, and try to remember everything. Oh well... Aggregator summary Photos are here. Had a great time seeing more of the country. Seattle was nice, albeit a bit wet. Portland was also nice. We've now covered the whole West Coast. Read on, Macduff!

It's not a Holiday Weekend unless it involves us putting heaps of miles on the car. This time, we're going to Kirkland, where I'll work from for the week. The route outlined on this image isn't quite the route we're going to take, we're going to camp amongst the redwoods on Saturday night, and stop off in Portland on Sunday night. On Friday night we'll stop where we drop, which we expect will be Ukiah. The following weekend, on the way back, we'll take a more direct route, which will probably be closer to what's on the map.